Skip to content

Better mini sitemap for the chatbot#1337

Merged
nrichers merged 8 commits into
mainfrom
nrichers/sc-16170/better-mini-sitemap-for-the-chatbot
May 24, 2026
Merged

Better mini sitemap for the chatbot#1337
nrichers merged 8 commits into
mainfrom
nrichers/sc-16170/better-mini-sitemap-for-the-chatbot

Conversation

@nrichers
Copy link
Copy Markdown
Collaborator

@nrichers nrichers commented May 20, 2026 •
edited
Loading

Pull Request Description

What and why?

This PR should improve chatbot responses by adding a product UI to docs map for RAG alongside the existing AGENTS.md docs index.

Response quality improvements that depend on docs RAG should be across the board, but I am hopeful that settings-related questions especially will improve, as settings 1) cover a large surface area in the UI and 2) our docs content architecture has grouped settings info with the human-centric tasks the settings belong to.

Summary of changes

  • Adds site/scripts/generate_chatbot_product_map.py to build a product-aligned map from frontend routes/help links to documentation URLs and H2/H3 section hints (committed as site/llm/chatbot-product-map.md).
  • Wires the generator into site/llm/render.sh and CI so the map and updated AGENTS.md are copied into site/llm/_llm-output/ for LanceDB ingestion.
  • Includes the "Using the documentation" hub page in the LLM corpus by excluding only contributor/style-guide pages under about/contributing/ (Quarto does not re-include files after a directory exclusion).
  • Extends AGENTS.md with a Product UI mapping section describing when Valerie should use the map vs docs-by-topic navigation.
  • Vendors frontend route/help-link data in site/llm/chatbot-product-map-frontend-snapshot.json so CI can build the map without a validmind/frontend checkout.

Fixes sc-16170 — Better mini sitemap for the chatbot

How to test

  • python3 -m unittest discover -s site/scripts -p 'test_generate_chatbot_product_map.py' -v
  • python3 site/scripts/generate_chatbot_product_map.py → git diff --exit-code site/llm/chatbot-product-map.md
  • make render-llm locally
  • Confirm site/llm/_llm-output/chatbot-product-map.md and site/llm/_llm-output/about/contributing/using-the-documentation.md exist; no validmind-community.md or style-guide/ under contributing

To refresh the vendored frontend snapshot after product UI link changes: make -C site refresh-chatbot-product-map (requires local validmind/frontend checkout).

What needs special review?

  • CI map verification and whether the vendored frontend snapshot workflow is clear enough for maintainers (site/llm/README.md).
  • LLM corpus exclusions under about/contributing/ — confirm only contributor/style-guide pages are omitted.

Dependencies, breaking changes, and deployment notes

  • No PR dependencies.
  • Post-merge: LanceDB must re-ingest the LLM corpus for the new map to affect Valerie in production.
  • Refresh chatbot-product-map-frontend-snapshot.json when frontend routes or helpLink values change.

Release notes

Internal — not externalized in release notes.

Checklist

  • What and why
  • Screenshots or videos (Frontend) — N/A
  • How to test
  • What needs special review
  • Dependencies, breaking changes, and deployment notes
  • Labels applied (internal)
  • PR linked to Shortcut
  • Unit tests added (Backend)
  • Tested locally
  • Documentation updated (if required)
  • Environment variable additions/changes documented (if required) — N/A

@nrichers
Add chatbot product-to-docs map for Valerie RAG (sc-16170)
c6388d8 
Generate a product-aligned mini sitemap from frontend routes and help links,
wire it into LLM render output, include the docs IA hub page in the corpus,
and verify the artifact in CI.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

Pull requests must include at least one of the required labels: internal, highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

@nrichers nrichers changed the title Better mini sitemap for the chatbot (sc-16170) Better mini sitemap for the chatbot May 20, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

Pull requests must include at least one of the required labels: internal, highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

@nrichers nrichers added the internal Not to be externalized in the release notes label May 20, 2026
nrichers added 3 commits May 19, 2026 18:56
@nrichers
Vendor frontend snapshot for chatbot product map (no CI frontend chec…
6aba415 
…kout)

Store extracted routes and help links in site/llm/chatbot-product-map-frontend-snapshot.json
so CI builds the map without validmind/frontend access. Refresh locally with
make -C site refresh-chatbot-product-map when product UI links change.
@nrichers
Document LLM corpus and frontend snapshot in site/llm/README.md
8bace69 
Give maintainers a single entry point for render-llm, product map artifacts,
and when to refresh the vendored frontend snapshot.
@nrichers
Fix nondeterministic chatbot product map generation for CI.
8702620 
Sort doc paths and related-doc suggestions so Linux and macOS produce the
same map, regenerate the committed artifact, and use unittest discover to
avoid the stdlib site module import collision.
@nrichers nrichers requested a review from kam-validmind May 20, 2026 17:41
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 21, 2026

Lighthouse check results

⚠️ WARN: Average accessibility score is 0.87 (required: >0.9) — Check the workflow run

Show Lighthouse scores

Folder depth level checked: 0

Commit SHA: 51ee46d

Modify the workflow to check a different depth:

  • 0: Top-level navigation only — /index.html, /guide/guides.html, ...
  • 1: All first-level subdirectories — /guide/.html, /developer/.html, ...
  • 2: All second-level subdirectories — /guide/attestation/*.html, ...
Page Accessibility Performance Best Practices SEO
/developer/validmind-library.html 0.85 0.68 1.00 0.82
/get-started/get-started.html 0.85 0.70 1.00 0.73
/guide/guides.html 0.85 0.69 1.00 0.82
/index.html 0.93 0.68 1.00 0.82
/releases/all-releases.html 0.86 0.69 1.00 0.73
/support/support.html 0.91 0.69 1.00 0.82
/training/training.html 0.85 0.61 0.96 0.73

kam-validmind
kam-validmind approved these changes May 22, 2026
View reviewed changes
Comment thread site/llm/chatbot-product-map-frontend-snapshot.json Outdated
Comment thread site/llm/chatbot-product-map-frontend-snapshot.json Outdated
nrichers added 3 commits May 22, 2026 18:45
@nrichers
Merge branch 'main' into nrichers/sc-16170/better-mini-sitemap-for-th…
8b34e52 
…e-chatbot

Resolve Makefile and render.sh conflicts (keep chatbot map targets; drop
retired test-descriptions). Fix settings index route hint and refresh
frontend snapshot per PR review.
@nrichers
Address comments from Kam
b02d4f1
@nrichers
Merge branch 'nrichers/sc-16170/better-mini-sitemap-for-the-chatbot' …
828b34e 
…of github.com:validmind/documentation into nrichers/sc-16170/better-mini-sitemap-for-the-chatbot
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 23, 2026

PR Summary

This PR introduces significant enhancements to the documentation build and validation process by integrating a chatbot product map generator into the CI/CD workflow. The key changes include:

  • Updates to the GitHub Actions workflow to:

    • Run the chatbot product map generator script.
    • Verify that the generated markdown and JSON snapshot files are in sync with the committed files by using a git diff exit code check.
    • Execute Python unit tests to validate the map generator functionality.
    • Extend validation by checking for the presence (or absence) of specific files in the LLM output directory.

  • Modifications to the Makefile to add and update targets such as generate-chatbot-product-map and refresh-chatbot-product-map, facilitating manual regeneration and refreshing of the chatbot product map as frontend routes or help links change.

  • Updates to documentation files (e.g., AGENTS.md and the LLM corpus docs) to describe how the chatbot product map is integrated, its purpose, and instructions on maintaining it when product routes or related configurations are updated.

  • A comprehensive Python script (generate_chatbot_product_map.py) that:

    • Extracts front-end route and help link data from the ValidMind frontend repository (or from a committed snapshot if the frontend is not available),
    • Parses and correlates the extracted data with documentation paths,
    • Generates a detailed markdown map that includes primary and related documentation links for each product route.

  • Addition of a unit test file (test_generate_chatbot_product_map.py) to verify key functionalities of the map generator including alias resolution, parsing of documentation references, path mapping from HTML to qmd, heading extraction, and a roundtrip test of the frontend snapshot.

Overall, these changes aim to automate the verification of documentation alignment with the product UI, improve the developer experience, and ensure that changes in the frontend that affect help links or routes are consistently reflected in the documentation corpus.

Test Suggestions

  • Run the full CI/CD pipeline to ensure that the new GitHub Actions workflow steps execute without errors.
  • Manually execute the command 'make generate-chatbot-product-map' to check that the product map is correctly generated and updated.
  • Run the Python unit test suite via 'python3 -m unittest discover -s site/scripts -p 'test_generate_chatbot_product_map.py' -v' to verify that all tests pass.
  • Simulate frontend changes by modifying a route or help link, then run the '--from-frontend' flag to ensure that the snapshot and subsequent generated map are updated accordingly.
  • Perform a git diff check after running the map generation to confirm that the committed files match the expected output.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 23, 2026

Validate docs site

✓ INFO: A live preview of the docs site is available — Open the preview

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 23, 2026

Lighthouse check results

✓ INFO: No site pages to audit in this PR.

Commit SHA: 828b34e

@nrichers nrichers merged commit d245dd5 into main May 24, 2026
7 checks passed
@nrichers nrichers deleted the nrichers/sc-16170/better-mini-sitemap-for-the-chatbot branch May 24, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kam-validmind kam-validmind kam-validmind approved these changes

Assignees

No one assigned

Labels

internal Not to be externalized in the release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nrichers @kam-validmind